Summary

Implements endpoint stickiness on retries to improve retry reliability and reduce unnecessary endpoint cycling. This PR differentiates between network errors (which should try a different endpoint) and HTTP errors (which should retry the same endpoint).

Related Issues:

• Partially addresses 渠道返回 503/400 错误时，系统在原渠道重复重试而非切换至其他备用渠道 #629 - System was repeatedly retrying the same provider instead of switching to backup providers when encountering 503/400 errors. This PR improves the retry logic by implementing intelligent endpoint selection based on error type.
• Follow-up to Fix provider exhaustion after model redirect (refs #629) #633 - Builds on the provider exhaustion fix by adding endpoint-level retry intelligence

Problem

The previous retry logic used attemptCount to cycle through endpoints, which caused two issues:

1. Network errors and HTTP errors were treated the same: Both would advance to the next endpoint, even when HTTP errors (like 500) might be transient and worth retrying on the same endpoint
2. Endpoint wrap-around: When endpoints were exhausted, the system would wrap around to the first endpoint again, potentially retrying already-failed endpoints

This led to suboptimal retry behavior where:

• Transient HTTP errors would unnecessarily cycle through all endpoints
• Network connectivity issues wouldn't properly advance to alternative endpoints
• Endpoint exhaustion could result in infinite retry loops

Solution

Introduced currentEndpointIndex variable to track endpoint selection independently from attemptCount, enabling different retry behaviors based on error category:

• SYSTEM_ERROR (network failures like ECONNREFUSED, ETIMEDOUT): Advances to next lowest-latency endpoint
• PROVIDER_ERROR (HTTP 4xx/5xx responses): Maintains same endpoint across retries (sticky behavior)
• Endpoint exhaustion: Stays at last endpoint without wrap-around using Math.min(currentEndpointIndex, endpointCandidates.length - 1)

Changes

Core Changes

• src/app/v1/_lib/proxy/forwarder.ts (+25/-2):
  • Added currentEndpointIndex variable to track endpoint selection independently (line 283)
  • Modified endpoint selection logic to use currentEndpointIndex instead of attemptCount (lines 320-326)
  • Added endpoint advancement logic for SYSTEM_ERROR only (lines 796-804)
  • Added no wrap-around protection with Math.min() clamping (line 325)
  • Added detailed inline comments explaining the stickiness behavior

Supporting Changes

• tests/unit/proxy/proxy-forwarder-retry-limit.test.ts (+397/-19):
  • Added 4 new comprehensive test cases:
    • SYSTEM_ERROR: should switch to next endpoint on each network error retry
    • PROVIDER_ERROR: should keep same endpoint on non-network error retry
    • SYSTEM_ERROR: should not wrap around when endpoints exhausted
    • mixed errors: PROVIDER_ERROR should not advance endpoint index
  • Updated 4 existing tests to use SYSTEM_ERROR for endpoint switching scenarios
  • All 10 proxy-forwarder-retry-limit tests pass

Testing

Automated Tests

• Added 4 new test cases validating endpoint stickiness behavior
• Updated 4 existing tests to properly use SYSTEM_ERROR for endpoint switching
• All 10 tests in proxy-forwarder-retry-limit.test.ts pass
• Test coverage includes: network errors, HTTP errors, endpoint exhaustion, and mixed error scenarios

Manual Testing Scenarios

1. Network error scenario: Verify SYSTEM_ERROR advances through endpoints (ep1 → ep2 → ep3)
2. HTTP error scenario: Verify PROVIDER_ERROR stays on same endpoint (ep1 → ep1 → ep1)
3. Exhaustion scenario: Verify no wrap-around when endpoints exhausted (ep1 → ep2 → ep2 → ep2)
4. Mixed error scenario: Verify correct endpoint progression with alternating error types

Checklist

• Code follows project conventions
• Self-review completed
• Tests pass locally (all 10 tests passing)
• Documentation updated (inline comments added)
• No breaking changes

Description enhanced by Claude AI

🤖 Generated with Claude Code

Greptile Overview

sequenceDiagram
    participant Client
    participant ProxyForwarder
    participant Endpoint1
    participant Endpoint2
    participant Endpoint3

    Note over ProxyForwarder: currentEndpointIndex = 0
    Note over ProxyForwarder: attemptCount = 0

    Client->>ProxyForwarder: send(request)
    ProxyForwarder->>Endpoint1: attempt 1 (index=0)
    Endpoint1-->>ProxyForwarder: SYSTEM_ERROR (network)
    Note over ProxyForwarder: currentEndpointIndex++ (0→1)
    Note over ProxyForwarder: attemptCount++ (0→1)
    Note over ProxyForwarder: wait 100ms

    ProxyForwarder->>Endpoint2: attempt 2 (index=1)
    Endpoint2-->>ProxyForwarder: PROVIDER_ERROR (HTTP 500)
    Note over ProxyForwarder: currentEndpointIndex stays at 1
    Note over ProxyForwarder: attemptCount++ (1→2)
    Note over ProxyForwarder: wait 100ms

    ProxyForwarder->>Endpoint2: attempt 3 (index=1, sticky)
    Endpoint2-->>ProxyForwarder: SYSTEM_ERROR (network)
    Note over ProxyForwarder: currentEndpointIndex++ (1→2)
    Note over ProxyForwarder: attemptCount++ (2→3)
    Note over ProxyForwarder: wait 100ms

    ProxyForwarder->>Endpoint3: attempt 4 (index=2)
    Endpoint3-->>ProxyForwarder: Success (200 OK)
    ProxyForwarder-->>Client: Response